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ABSTRACT 

This study used structural equation modeling and multilevel 
modeling approaches for purposes of simultaneous study of individual and group 
change patterns on three waves of two longitudinally assessed domains. This 
study illustrates a few of the dual approaches to the analysis of covariance 
structures as they relate to the same individual growth model and the same 
data. LISREL (structural equation modeling) and hierarchical linear modeling 
(HLM) were used. The investigation studied whether individual change over time 
in mathematics and language differs from student to student and whether the 
individual growth parameters of each of the two domains were related within 
domain. The study used panel data from the Louisiana State Department of 
Education for 3 waves of students tested in grades 4, 6, and 7: (1) 50,907; 

(2) 47,003; and (3) 50,157. Assessments from the state's testing program were 
administered in each of the three grades. Complete records for all 3 grades 
were available for 26,051 students, 11,627 of whom were African American. The 
study sheds light on the understanding of learners from the two ethnic groups 
and shows how they develop mastery in mathematics and language as they 
progress through school. The application of covariance structure analysis and 
growth curves to the study of growth in student academic achievement provides 
an avenue for an in-depth analysis of two academic areas in an available data 
set. The statistical techniques used in this research, LISREL and HLM methods, 
have a number of extensions that can be used in various research environments 
because they can accommodate any number of data points (waves) of longitudinal 
data. Four appendixes contain descriptive statistics from the study. (Contains 
2 figures, 3 tables, and 50 references.) ( SLD) 
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A Study of Individual Patterns of longitudinal Academic Change: Exploring the Structural 
Equation Modeling (SEM) and Hierarchical Linear Modeling (HLM) 

While many theorists have presented models to describe growth and change, these models 
are infrequently tested with data (Magnusson, 1 985). It is apparent that lack of familiarity with many 
quantitative methods for estimating learning growth curves appears to be a major obstacle to the 
empirical testing of growth models (Burchinal & Appelbaum, 1991). Bryk and Raudenbush ( 1 992) 
amplified the same problem by noting that research on change has been plagued by inadequacies in 
conceptualization, measurement, and design and has long perplexed behavioral scientists. In many 
situations, instruments used to assess the subjects are developed for fixed points in time, yet 
individual academic growth is dynamic. These instruments have not adequately captured individual 
differences in the rate of change. The study of change requires more than two waves of data but 
frequent studies have utilized only two data points and are thus not able to adequately address the 
issue of growth (Bryk & Raudenbush, 1 987; Bryk & Weisburg, 1 977; Rogosa, Brandt, & Zimowski, 
1982). When there are only two waves of data on each subject, there is no way to know the exact 
shape of individual growth over time (Willett, 1988). It has also been stressed that data from two 
time points and the difference score are less than optimal for the study of change but three or more 
waves of data are preferable (Olweus & Alsaker, 1991). 

The difference score that was initially employed and continues to be used as a measure of 
change because of the concentration of two- waves measurement has restrictive assumptions and its 
continued use as a measure of change has been condemned by many researchers (Cronbach, Furby 
1970;Lord, 1963; O’ Connor, 1972; Thorndike, 1966). These researchers have instead recommended 
other statistical techniques of evaluating change. 



O 

ERIC 



2 



3 



Why study change in education? A focus on the study of change enables an in-depth 
investigation of how key elements of learning in and other variables exert an influence on student 
achievement outcomes. A study of change in education lends itself to an indepth evaluation of the 
extent differences in schooling experiences; in particular, differences in classroom environment and 
instructional quality, contribute to the development of interindividual differences in achievement. 

Students are enrolled in schools so that they can grow academically and educationally, 
develop, and change. It is the measurement of these changes and the investigation of their 
relationship to supporting activities in the classroom and the resources provided by the school that 
empirical investigations ought to focus on (Willet, 1988). The study of this change in education is 
important because it is through change that the effectiveness of a curriculum can be assessed and 
improved. 

The study of individual academic change has a relatively long history. The growth in the 
measurement of change has been gradual and the earlier problems that faced the adequate 
measurement of change continue to be addressed. In recent research on individual change, 
investigators have used individual growth modeling in order to make use of the enormous volume 
of multiwave data available in academic and related institutions, while providing better methods for 
investigating interindividual differences in change (Bryk & Raudenbush, 1 987; Rogosa et al., 1 982; 
Rogosa & Willett, 1985; Sayer & Willett, 1998; Willett, 1988; Willett & Sayer,1994, 1996). 

Recent studies (Raudenbush, 1995) have revealed that widely available software can be 
adapted to provide maximum likelihood (ML) estimates for a general class of multilevel covariance 
structure models if the data are balanced — i.e., equal numbers of students in each of the many 
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schools, thus ensuring that every level-2 unit has same number of level- 1 units. Studies of individual 
change are increasingly employing a combination of individual growth trajectories and structural 
equation modeling (SEM), while capitalizing on the unique strengths each of these procedures 
offers. SEM encompasses an entire family of models known by many names, among them 
covariance structure analysis, latent variable analysis, confirmatory factor analysis, and often simply 
LISREL analysis (Linear Structural RELations - the name of one of the more popular statistical 
software packages). SEM is a statistical methodology that takes a confirmatory perspective to the 
multivariate analysis of a structural theory bearing on some underlying phenomenon (Byrne, 1998). 
An equation which relates the dependent (Y) and independent(X) variable, such as Y = a + bX, is 
a structural equation, and the constants a and b are structural coefficients. When two or more 
equations simultaneously describe the set of variables under consideration, such equations are 
considered as structural equation models. SEM generally employs the maximum likelihood method, 
which is a large-sample procedure and is unlikely to behave well with small sample sizes in a 
multiple group perspective (Burstein, Kyung-Sung & Delandshere, 1989). Recently, pioneering 
researchers have shown how the analysis of change can be conducted conveniently by the methods 
of covariance structure analysis (Tisak & Meridith,1990; Sayer & Willett, 1998; Willett & Sayer, 
1994, 1996). 

The application of covariance structure analysis techniques in research subsumes more 
traditional approaches to the analysis of panel data, such as repeated measures analysis of variance 
(ANOVA) and multivariate analysis of variance (MANOVA) (Joreskog & Sorbom, 1989; Meridith 
& Tisak, 1990; Rao, 1958; Tucker, 1958). For this study, the individual change aspect was 
represented through a two-level hierarchical model (“multilevel”). At level 1, each person’s 



development is represented by an individual growth trajectory that depends on the unique set of 
parameters. In level 2, the level 1 growth parameters become the outcome variables, where they 
depend on some person-level characteristics. The multiple observations recorded for each individual 
in the study provide a ‘hierarchy’ which can be adequately processed by a multilevel data analysis 
technique. 

Multilevel analysis involves estimating growth curves for multiple observations in the first 
phase and testing the covariation between the estimated indices of growth curve analysis and 
hypothesized predictors or outcomes of the change process in the second phase (Bryk & 
Raudenbush, 1992; Muthen, 1997; Sayer & Willett, 1998; Willett & Sayer, 1994, 1996). The 
multilevel covariance structure analysis model is a flexible procedure and as such an attractive 
analytical tool for a variety of SEM analyses that can be used to investigate growth and development 
among variables of interest with multilevel data. This study, with availability of panel data, and 
particularly for studies of individual growth, will demonstrate how covariance structure models can 
be set up for hierarchical data (observations nested within person) and how these models can be 
analyzed by traditional SEM software, such as LISREL. 

Longitudinal studies occupy an important place in the psychological and social sciences 
research realm. In these studies the same individuals are repeatedly measured on a number of 
targeted variables over a series of important time points (Hedleker, Gibbons, 1 997). However, there 
has been a real struggle among researchers for some time over concepts such as hierarchical nested 
observations, intra-class correlations, the unit of analysis, and random rather than fixed effects 
(Duncan, Duncan, Alpert, Hops, Stoolmiller, & Muthen, 1997). The study of change, like other 
complex studies, has been slowed by lack of a complete and stand-alone statistical package that has 
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the capacity and capability of handling all the univariate and multivariate statistical data analysis 
requirements. Most data for the analysis of change must go through a series of preprocessing stages 
before they can be utilized to analyze change. 

The conceptual framework guiding this study is based on the recent emerging approach to 
the measurement of change, the latent growth curves within structural equation modeling such as 
LISREL (Duncan etal. 1999; Duncan & Duncan, 1990; McArdle & Epstein, 1987; Patterson, 1993; 
Sayer & Willett, 1998; Stoolmiller, 1994; Stoolmiller, Duncan, Bank & Patterson, 1993; Willett & 
Sayer, 1994, 1996). McArdle and Epstein (1987) defined the latent growth curve model (LGM) as 
a “longitudinal model that includes correlations, variances, and means.” The inclusion of means in 
these models make them more similar to repeated-measures ANOVA and MANOVA. Latent growth 
model and general multivariate growth models try to describe the way the individual develops. 
Studies about change have been cross-sectional, with the center of investigation being mean level 
changes across different groups. Over the past twenty five years, researchers have called for the 
development of appropriate methods for the analysis of longitudinal data (Mehta & West, 2000). 

In longitudinal research, investigators often measure multiple variables at multiple points in 
time and are interested in investigating individual differences in patterns of change on those variables 
(MacCallum, et al., 1997; McArdle & Aber, 1990). MacCallum, et al. (1997), McArdle and Aber 
(1990) study focused on the relationships between patterns of change on different variables while 
showing how the multilevel modeling framework, which is often used to study univariate change, 
can be extended to the multivariate case to yield estimates of covariance of parameters, which 
represent aspects of change on different variables. 



This study utilizes structural equation modeling and multilevel modeling approaches for 
purposes of simultaneous study of individual and group change patterns on three waves of two 
longitudinally assessed domains. Recent studies (Raudenbush, 1995) have revealed that widely 
available software can be adapted to provide maximum likelihood (ML) estimates for a general class 
of multilevel covariance structure models if the data are “balanced”— i.e., equal numbers of students 
in each of the many schools, thus ensuring that every level-2 unit has same number of level- 1 units. 

Schmidt ( 1 969) developed software to compute maximum likelihood (ML) estimates for two- 
level data for a balanced design. Mcdonald & Goldstein (1989) provided theory for ML estimation 
for unbalanced models that incorporate both level- 1 and level-2 variables showed that software such 
as EQS (Bentler, 1983), LISCOMP (Muthen,1987) and LISREL (Joreskog & Sorbom, 1996) can be 
implemented with little or no modification to the software for the analysis of multilevel data. Bryk 
and Raudenbush (1987) advanced strengths of HLM in its ability to make predictions and the fact 
that the HLM model draws on strengths are available in the data. 

Purpose 

The purpose of this study was to demonstrate a few of the dual approaches to the analysis of 
covariance structures as they relate to the same individual growth model and based on the same data. 
More specifically, using both the LISREL 8 (including PRELIS 2) and SAS PROC MIXED (SAS 
Institute, 1996). The investigation was based on a study of whether individual change over time in 
mathematics and language differs from student to student and if the individual growth parameters 
of each of the two domains were related within domain. The study was guided by the following 
specific research questions: (a) are the growth parameters (intercepts and slopes) in mathematics and 
language related within each domain? (b) is the pattern of interrelationships, among the individual 




7 



8 



achievement growth parameters, the same for African American and White students? (c) are there 
discernible patterns in variability in academic growth parameters within each ethnicity? 

Methodology 
Sampling Procedures 

This study used panel data drawn from the Louisiana State Department of Education (LDE) 
school data files. The subset of students involved was obtained as follows. Of all the elementary 
school students in the LDE data files, only those who attended public schools and were of African 
American and White ethnic group origins were sampled. The sampled students were tested on the 
Norm Referenced Tests (NRTs) in grade 4, 6 and 7. Wave one had 50,907 students (African 
Americans=24,030, Whites=26,872), wave two had 47,003 students (African Americans=22,262, 
Whites=24,741) while the third and last wave had 50,157 students (African Americans=23,982, 
Whites=24,536). The subsets of students who had complete records for grades 4, 6 and 7 were 
26,051 (African Americans=l 1,627, Whites=l 4,424). 

Instrumentation and Measurement 

The Iowa Tests of Basic Skills (ITBS) and Norm-Referenced Tests (NRTs) as part of the 
Louisiana Educational Assessment Program (LEAP), was utilized. The two domains utilized in this 
investigation (language and math) were average composites of their respective constituents. Math 
subscales were math concepts/ estimation and math problem solving/data interpretation while 
language subscales were spelling, capitalization, punctuation, and usage and expression. The NRT 
measure is a multiple choice scale for mathematics and language domains and allow the educators 
to compare individual and group performance results with a national norm. These tests indicate how 
a given student’s knowledge or skill compares with others’ in the norm group. Reliability data for 
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the ITBS meet stringent psychometric standards where the ITBS Complete Battery average test 
reliabilities (K-R 20) for grades 3 through 8 are 0.86 and 0.87 for the fall and spring, respectively. 

Data Analysis Procedures 

This study adopted a two-stage road-map of the data analysis procedure as provided in the 
individual growth curves analysis of Singer (1998) and covariance structure analysis technique of 
both Sayerand Willett (1998) and Willett and Sayer (1994, 1996) for single and double populations. 
First, a series of preliminary data analyses was conducted to check on the normality, skewness, and 
kurtotic nature of each of the three waves of data to gain familiarity and knowledge of the data at the 
individual level (See Appendix A). Ordinary least squares (OLS) fitted trajectories summarizing 
observed growth patterns for both mathematics and language between grade 4 and 7 for the 
subsample of 27 (See Appendix B) selected students from both ethnic groups was completed 
(Appendices, C and D). 

In the study of change patterns in student academic achievement, over time, the analysis was 
conducted in two levels. At level 1 (within person), the curve fitting techniques to describe growth 
events such as the effect of student grade level on mathematics and on language achievement were 
applied. This level involves fitting, to each individual, a particular curve that is a function of time 
(grade). In the second level (between-person), comparison of the patterns of the growth parameters 
was made. The different student background characteristics was presented through the summary 
descriptions of means of the individual curve coefficients gleaned from the first level analysis. The 
multilevel data analysis techniques carry out such analysis at two levels simultaneously (Bryk & 
Raudenbush, 1992; Kaplan & Elliott, 1997; Yang & Goldstein, 1996). The individual growth model 
was evaluated in line with the tenets of the classical test theory approach where the observed score 
is distinguished from the true score. Table 1, presents the sample mean vectors and covariance 
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Table 1: Estimated Means and Covariances for Three waves of Mathematics and Language 
Achievement Scores at grades 4, 6, and 7 for (a) African American (AA) students (n= 10,724), (b) 
White students (n=l 3,578). 



AA 


Mathematics 




Language 






Grade 


4 


6 


7 


4 


6 


7 


Means 


186.28 


208.06 


216.76 


189.72 


213.53 


226.84 


Covariances 


256.24 














176.34 


358.01 












225.85 


316.87 


553.50 










214.68 


185.73 


238.85 


404.99 








222.64 


319.37 


369.70 


355.48 


726.40 






226.07 


299.66 


427.87 


349.59 


545.47 


811.31 


N=1 0,724 














WHITE 


Mathematics 




Language 






Grade 


4 


6 


7 


4 


6 


7 


Means 


204.50 


231.01 


246.01 


206.66 


237.53 


251.73 


Covariances 


411.12 














329.49 


541.72 












372.02 


481.18 


687.09 










322.37 


308.73 


358.42 


574.12 








355.65 


446.36 


492.71 


517.45 


901.85 






350.66 


421.75 


541.20 


488.67 


682.69 


959.18 



N=1 3,578 
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matrices for language and mathematics and for the two groups of leamers-African American and 
White students respectively. The data in the table was utilized in the computation of individual 
growth parameters for the two groups of learners. 

Random Coefficient Regression Analysis (Hierarchical Linear Modeling) 

The results of the covariance structure analysis were computed and compared with those 
derived from a hierarchical linear modeling approach, utilizing the SAS PROC MIXED routine, as 
detailed in the works of Littell, Milliken, Stroup, and Wolfmger (1996), and Singer (1998). In 
utilizing this approach, individual growth models for mathematics and language were treated as 
linear functions of time with the individual intercepts and slopes treated as random. Using this 
technique (hierarchical/random coefficient modeling), “an unconditional linear growth model” with 
a simple two-level model was considered, in which the level-1 model is a linear individual growth 
model, and the level-2 model expresses variation in parameters from the growth model as random 
effects unrelated to any person-level covariates/predictors. The parameters in level- 1 (within 
person) model used 71 and the parameters in the level-2 (between person) model used p. The level- 1 
and level-2 models were then written as: 



which were written in combined form as: 




Yij= [Poo+ Pio Time]jj + [u^ + u^Time^ + r (j ], 



As can be seen above, the multilevel model was expressed as the sum of two parts: a fixed 
part, which contains two fixed effects (for intercept and for the effect of TIME) and a random part, 



which contains three random effects (for the intercept, the TIME slope, and within person residual 
ry). The time variable for this study was grade level and it appeared in the model line as the predictor 
for both mathematics and language. The treatment of the intercept and slopes as random effects can 
be changed, and also the covariates (predictors) of the level-2 components can be introduced 
depending upon the nature of the particular research question. 

Results 

The major findings of the study showed that: 1) students vary significantly in knowledge of 
mathematics at entry into grade 4 and that White students overall initial status in mathematics was 
higher than that of African American students, 2) language intercepts for the two groups were 
statistically significant, signifying language knowledge differences at grade 4, 3) the mathematics 
overall slopes for the two groups of learners were positive and significantly different from zero, 4) 
language overall rates of learning within ethnicity were significantly different from zero, 5) the 
correlation coefficients of the slope and initial status for each domain and within each ethnicity were 
not statistically significant, 6) variance estimates for language and mathematics slopes were 
significantly different from zero and showed variance increases at lower grade levels as students 
advance in school from grade 4 through grade seven, and 7) the results of LISREL analyses and those 
of Hierarchical Linear Model (random coefficient regression analysis) were similar in intercepts and 
slopes both within domains and across learning groups. A LISREL model that reasonably fit the 
data well showed marked different results from those of HLM. The results described in the above 
section are summarized in Tables 2 and 3 presented in the following section. 

In this study, LISREL model 2 maximum likelihood estimates of the population means of 
true intercept and true slope in both mathematics and language for both the African American and 
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White students groups were reported. 

The entries in the first two rows of Table 2 for Model 2 estimate the African American 
population means of true intercept (188.96, p < 0.05) and true slope (10.01, p <0.05) for 
mathematics. The estimated population means of true intercept and true slope for language are 
189.02 (p < 0.05) and 12.42 (p < 0.05), respectively. The true intercept and true slope for the 
respective domain describe the average trajectory of true change in the dependent variable. On 
average, African American students’ true mathematics scores increase by 10.01 per year while true 
language scores increase by 12.42 per year. 

Table 3 presents parameter estimates and model fitting for mathematics and language scores 
for the White students. As was the case with the African American model fitting, model 2 was 
adopted for each domain. An inspection of the parameters in the table show that all the intercept 
parameters were statistically significant. Entries in the first two rows of Table 4.10 for Model 2 
estimate the White population means of true intercept to be 204.04 (p < 0.05) and true slope to be 
13.80 (p <0.05) for mathematics. 

The estimated population means of true intercept and true slope for language were 207.09 
(p< 0.05) and 1 5.00 (p< 0.05), respectively. These growth parameters describe the average trajectory 
of true change in the dependent variable. On average, White students’ true mathematics scores 
increase by 13.80 per year while true language scores increase by 15.00 per year. Students’ 
knowledge in both mathematics and language improved over time, and more rapidly in language 
than in mathematics. 

In both the African American and the White samples, slope parameters were all positive 
and statistically significant. The domain respective intercepts are initial average achievement 



Table 2: Fitted Models For Interindividual differences in Change in Mathematics and Language 
in the African American Sample 





Maximum Likelihood Estimates 




Parameter 


Mathematics 


Language 






Model 1 


Model 2 


Model 1 


Model 2 


[i 0p (Intercept [I]) 


186.59* 


188.96* 


189.52* 


189.02* 


H,p (Slope [S]) 


10.26* 


10.01* 


12.31* 


12.42* 


o^Q 2 (Intercept Variance) 


312.33* 


215.86* 


619.72* 


354.22* 


o^, 2 (Slope Variance) 


51.03* 


27.77* 


103.58* 


62.60* 


° 7 i 07 ii (I-S Covariance) 


-53.31* 


0.12 


-104.35* 


0.48 


df 


3 


1 


3 


1 


x 2 


324.78* 


8.56* 


742.40* 


41.45* 


Goodness of Fit Index (GFI) 


.985 


1.000 


.960 


1.000 


Normed Fit Index (NFI) 


.960 


.999 


.952 


.997 


Comparative Fit Index (CFI) 


.960 


.999 


.952 


.997 


Root-Mean-Square Error 
of Approximation (RMSEA) 


.100 


.026 


.152 


.061 



Note: N=1 0,724. Descriptions of the models are given in the text below 
* p< .05 
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scores at grade 4 adjusted for measurement error (Kline, 1 998). The intercept is a characteristics of 
the whole sample while the variance of the same, reflects the range of individual differences in the 
domain of interest around the intercept. The mean rate of change, on the other hand, reflects a group- 
level characteristic- its value indicates the average amount of occasion-to-occasion change in mean 
levels of the domain of interest (also adjusted for measurement error). The statistics provided by the 
slope(rate of change) presents information about the rate of individual differences in linear occasion- 
to-occasion changes over time. 

Tables 3 and 4 summarize the maximum likelihood estimates of the growth parameter matrix 
of individual differences in true change that were detected in mathematics and language. The 
variances of both mathematics and language were statistically significant. Thus, there is evidence 
of interindividual heterogeneity in true change in mathematics and language. Thus, students differed 
in their growth trajectories in these two domains. Correlation coefficients of intercepts and slopes 
within each domain were not statistically significant but they were both positive in direction. These 
coefficients show that students who had high initial language achievement scores showed greater 
rates of subsequent change. They tended to progress more rapidly in language over time. The same 
could be said about mathematics. However, the intercept was unrelated to the slope changes in the 
respective domain. Thus, where a particular student starts in an achievement domain is not 
necessarily related to his or her future growth (mean level) in the domain of interest. This conclusion 
is supported by the individual sample growth trajectories provided in Figures 1, 2, 3, and 4 
(Appendices C, D). 
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Table 3 : Fitted Models For Interindividual differences in Change in Mathematics and Language 
in the White Sample 





Maximum Likelihood Estimates 




Parameter 


Mathematics 


Language 






Model 1 


Model 2 


Model 1 


Model 2 


Hop (Intercept [I]) 


204.25* 


204.08* 


206.84* 


207.09* 


Hip (Slope [S]) 


13.75* 


13.80* 


15.08* 


15.00* 


O n0 2 (Intercept Variance) 


493.38* 


328.74* 


840.48* 


516.95* 


O n] 2 (Slope Variance) 


60.69* 


35.19* 


113.76* 


63.96* 


O K0Kl (I-S Covariance) 


-64.40* 


0.29 


-126.35* 


0.20 


df 


3 


1 


3 


1 


x 2 


199.55* 


39.24* 


584.95* 


23.32* 


Goodness of Fit Index (GFI) 


.992 


1.000 


.974 


1.000 


Normed Fit Index (NFI) 


.981 


.999 


.965 


.999 


Comparative Fit Index (CFI) 


.981 


.996 


.965 


.999 


Root-Mean-Square Error 
of Approximation (RMSEA) 


.069 


.053 


.120 


.041 



Note: N-l 3,578. Descriptions of the models are given in the text 
*p< .05 
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Table 4 presents a summary of the results of random coefficient regression analysis 

(hierarchical linear modeling) for both mathematics and language. The results show the presence 

of variability in the intercepts and slopes and the covariance-variances for the two groups of learners 

and within each domain. When compared with the earlier results of for the LISREL analysis, the 

(HLM) intercepts and slopes are exactly the same both within domains and across learning groups 

as was the case with LISREL model 1 . However, there are marked differences in the variance- 

covariance estimates for both language and mathematics, as presented in Tables 2, 3 and 4. This 

Table 4: A Summary of the Random Coefficient Regression Analysis (Hierarchical Linear 
model-HLM) Results for both Mathematics and Language. 





Intercept (I) 


Slope (S) 


Variance (I) 


Variance (S) 


o 2 


AA a 












Math 


186.60 


10.26 


123.19 


10.69 


125.30 


Language 


189.52 


12.31 


221.07 


9.75 


202.24 














WHITE 












Math 


204.25 


13.75 


278.00 


10.89 


127.42 


Language 


206.84 


15.08 


378.81 


9.02 


227.93 



a African American 

Note: All entries in table are significant at p < 0.05 

may be due to computational improvements within each software, differences in how each 
software handles missing values that may not be missing at random, the problem of 
multicollinearity or the differences could simply be an artifact of scale scores and equating 
processes. 
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Discussions/Significance 



This study is important in a number of ways. First, the study contributes to the expansion 
of literature brought about by research on the key components of the model of interest and the 
associated findings. Secondly, it sheds light on the understanding of two groups of learners from two 
ethnic groups and how the two learners develop mastery in mathematics and language as they 
graduate through school. Thirdly, the application of covariance structure analysis and growth curves 
to the study of growth in student academic achievement provides an avenue for an indepth analysis 
of two academic areas in an available data set. The use of longitudinal assessments in the 
identification process and assessment of outcomes offer several advantages over the traditional static 
cross-sectional assessment of learning outcomes. The employment of this technique shifts the focus 
from the assessment of mathematics and language achievement to learning and in effect leads to a 
more refined definition of learning problems and measurement of outcomes, which is a conceptual 
advantage over the current traditional approaches. 

The study about growth in academic achievement is significant in that a better understanding 
of the cognitive abilities of different groups of learners in different academic fields is realized. 
Achievement outcomes are normally collected at the end of a specified period in the student 
academic career. The use of longitudinal assessments to measure growth in academic achievement 
makes early detection of learning problems a reality in that the rates of learning can simultaneously 
be measured in mathematics and language to assess the degree to which skills are differentially 
developing. Ultimately, the value of the study pertains to the following: 

a) It provides a close examination of the trends and individual differences in mathematics and 
language and explores the effects of ethnicity on developmental trends that could go undetected due 
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to insufficient power by more traditional analyses such as ANOVA, MANOVA, (M)ANCOV A, etc, 
b) It draws similarities in model formulation between the traditional methods, such as regression 
analysis, and covariance structure analysis with a view of lessening the burden inherent in SEM and 
HLM technical aspects that would naturally close out potential users of important research findings 
of student academic growth, c) it provides a potential base for further research on the measurement 
of change in student academic achievement which eventually may lead to schools and school systems 
adopting of measures tailored to meeting specific needs of specific students or groups of students, 
d) it provides the research findings on academic growth to educators, parents, and Louisiana 
department of education, among other school stakeholders, for the benefit of the education in the 
state. 

From the measurement theory, research design and future practice perspectives, the statistical 
techniques employed in this research (the LISREL and HLM) methods have a number of extensions 
that can be utilized in various research environments due to their abilities to accommodate any 
number of data points (waves) of longitudinal data with more data leading to higher precision for 
the estimation of the individual growth parameters and greater reliability of the measurement of 
change. 

Missing Data 

The results of this investigation should be interpreted with some caution because of a number 
of factors that were beyond the control of the researcher. Important among these was missing data. 
More often than not, loss of subjects in longitudinal studies of students may result in the pattern of 
data loss that may not be random. Due to the rather large data set utilized in this study, a test of 
whether the patterns of missing data were random or systematic was not completed but an 



assumption was made that the missing cases in the data set were purely random and that missing 
data would not adversely affect the sample size. However, students who dropped out of school at 
each wave are perhaps more likely to come from families with particular characteristics (e.g., low 
SES, job instability of parents). This obviously can create problems with reliability of the data and 
the general izability of the results. Further, the growth parameters computed may not be adequately 
representative of the true change in achievement for the ethnic groups compared over time. It is also 
important to be cognizant of the fact that when the missing pattern is not random, there is no 
adequate statistical fix to remedy this problem. 

Though this study did not attempt to model the problem of missing data, it employed listwise 
deletion. The covariance matrix generated by listwise deletion will always be consistent, that is, 
positive semi-definite (Anderson and Gerbing, 1 984). However, if the pattern of missing data is not 
random, an inconsistent matrix - not positive definite, can result (Rovine , & Delaney, 1990). 
Despite the fact that listwise deletion can result in a positive semidefinite matrix, it is also known 
that this technique can present problems for tests of goodness of fit, unless the missing data are 
missing completely at random (Kaplan, & Elliott, 1997; Muthen, Kaplan, & Hollis, 1987). 

Though there have been advancements in statistical computing power, multivariate data are 
frequently hampered by missing values. The traditional and relatively old methods of dealing with 
incomplete data, that is, deletion (listwise, pairwise) for cases with incomplete information, 
substituting plausible values such as means, or regression prediction for missing values continue to 
be utilized. In this study, listwise deletion was used. With listwise deletion cases with missing 
observations on any variable in any analysis are excluded from all computations-thus a final sample 
includes only cases with complete records. Though the recent advances in theory and computational 
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statistics have produced flexible and powerful procedures with sound statistical bases (Likelihood- 
Based Estimation-Efficient Estimation-EM, Multiple Imputations-MI) (Cohen, & Cohen, 1983; 
Kline, 1998; Schafer, & Olsen, 1998; Rovine, & Delaney, 1990), the statistical processes involved 
are above the reach of many researchers who are faced with the problem of missing data on a daily 
basis. These computational statistical techniques are quite involved and may require equally 
demanding data preparation procedures which many users of secondary data analysis may see as a 
nuisance that should be avoided as much as possible. 

Furthermore, many techniques for handling missing data rarely account for the patterns of 
missing observations— whether random or systematic. This is a much bigger problem and compounds 
that of the proportion of the missing data. There is no clear guideline about how much missing data 
is too much. Cohen and Cohen (1983) suggested that 5% or even 1 0% missing data on a particular 
variable is not large. Irrespective of the method utilized in imputing missing values, the data set 
would still fail to provide accurate measures of variability if it does not account for missing-data 
uncertainty (Schafer, & Olsen, 1998). 

As discussed earlier, intercept changes in both language and mathematics and for the two 
groups of learners were unrelated to their respective slopes. This suggest that where a student starts 
in domain achievement is not necessarily related to his or her future growth in the domain of interest. 
Though this study did not investigate poverty among the two groups of interest, it is worth noting 
that poverty in the African American sample in Louisiana is much higher than that of White sample. 
This imbeddedness of poverty within any particular group translates into differential learning 
environments in terms of per capita learning resources made available at home, which subsequently 
impacts school learning and achievement. Though a number of individual growth patterns over time 



were shown in this study with each group, and when comparisons were made within group by SES 
levels, the total group effects of home and schooling were shown to sustain over time. Recent large 
scale reviews of the literature to identify both proximal and distal factors impacting student learning 
and achievement clearly document the importance of proximal factors that include both the school 
and the educational quality of the home environment (Wang, Haertel, & Walberg, 1993). 

African American and White students enter grade 4 with language and mathematics 
achievement differences. These differences are more than influenced by differing rates of poverty 
associated with race. However, the results reported here also suggest that proximal factors 
associated with school (i.e., differing teacher expectations, access to educational resources) may also 
differentially affect African American and White students. Both the mathematics and language 
intercept and slope variances were higher for White students than for African American students. 
These differences suggests that the effects of home and school learning environments witthin groups 
differ. The White sample in this study remained approximately normally distributed with both low, 
median and high achievers persisting through the schooling years. This may not be the case with 
African American students over a greater number of years when differential dropout rates might be 
expected. These rates might well be predicted by difficulties associated with early childhood 
learning experiences. Thus, shrinkage in differences in achievement between White and African 
American groups in the later years of schooling might well be expected by differential dropout rates. 
As well, greater variation in SES within these two groups might account for the greater heterogeneity 
in White student samples in later school years than in African American student samples (as shown 
in this study). 

It also seems important that factors that directly relate to proper and reliable assessment of 
student achievement in mathematics and language be observed. Royer (1990), stated that test using 
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multiple-choice items were measuring offline reasoning processes rather than online comprehension 
processes and extreme care must be observed when using these tests to make grade placement 
decisions, diagnosing reading difficulty, or assessing educational gain. Royer (1990) argued that 
standardized reading comprehension tests that utilize multiple-choice questions do not measure the 
comprehension of a given passage, but rather measures a reader’s world knowledge and his or her 
ability to reason and think about the content of the passage. For mathematics and language educators 
need to use multiple data points and multiple forms of assessments of students’ knowledge of 
mathematics and language other than relying only on the scores of standardized tests to evaluate 
students’ learning growth. Both reliability and validity of inferences about student learning and 
academic progress are enhanced with analyses of longitudinal data. 

It is important also that teachers have a better understanding of their students’ literacy 
development. This helps teachers to recognize patterns of behavior which suggests aspects of 
students’ development behavior out of what is provided in the curriculum. Knowledge of student’s 
literacy development accords teachers an opportunity to develop more flexible curricula to meet the 
changing needs of specific students or groups of students. 

The Louisiana School Effectiveness study (Teddlie, 1994; Teddlie & Stringfield, 1993) 
discussed areas in which school policies can positively affect teachers behaviors such as appropriate 
teacher selection and replacement, frequent personal monitoring of classroom behavior, support for 
teachers through direct assistance and in-service programs, and overall instructional leadership. 
These strategies lay a fertile ground for effectiveness in classroom instruction and management. 
Mendro (1998) discussed equity in student access to a quality education as regards the type of help 
to provide to students who have had an ineffective teacher in the past. Mendro (1998) stated that 



students who are placed with an ineffective teacher suffer long-term negative effects and there needs 
to be a policy issue put in place to allow for more equitable distribution of resources to enhance the 
quality of teaching and learning. In a reent study that aggregated data at the student level, Sanders 
and Horn (1998) found that ineffective teachers were ineffective with all students regardless of 
students’ prior levels of achievement while teachers of the highest effectiveness were generally 
effective will all students. Though Sanders & Horn (1998) found teacher effectiveness to be a 
dominant factor affecting student gains in academic achievement when compared to other classroom 
context variables (.e.g, class size, classroom heterogeneity), it seems important that schools 
recognize socioeconomic differences among students in th early years in considering more equitable 
distribution of educational resources, particularly good teachers. 

For future research, this study raised a number of important points to consider for future 
research. First, student language and math achievement change need more research to pinpoint 
exactly where differences arise within each domain and across ethnicity. Second, lower math 
achievement scores and rates of change, particularly for African American students needs more 
intense study. The National Center for Educational Statistics study showed that, on average blacks 
and Hispanics score lower than Whites on reading and mathematics at the end of grade 8 and that 
these differences do not increase over the high school years. Sanders and Horn ( 1 998) showed that, 
regardless of race, students who are assigned disproportionately to ineffective teachers are severely 
academically handicapped relative to students with other teacher assignment patterns. More research 
that links students’ academic records to those of their teachers seems in order. 

Third, the methodology of this study needs to be extended to ethnically diverse samples to 
further demonstrate its utility for investigating individual change over time. Studies using multi- 

24 




25 



domain analyses to further investigate the nature of differences that were observed in language and 
math parameters in this study, and whether these differences are maintained across different groups 
of learners are needed. 

Fourth, a replication of this study that uses the same measuring instrument across all 
measurement occasions, and a greater number of occasions, is recommended. This is preferred to 
using equating procedures such as vertical equating, with different tests. A greater number of data 
points (more waves) might also be quite informative. Such studies can yield information that has 
implications for understanding academic growth differences both within and between differing 
groups, and information that might be used for educational policy making, resource allocation and 
school intervention and improvement programs as well. In an era of educational policy making for 
greater school accountability, longitudinal studies can be used to better understand patterns of school 
change (or lack of change) over time. This seems particularly the case when such procedures are 
compared to more traditionally used procedures (i.e., pre and post test analyses from year to year). 
The data analysis procedures used in this study, and the attained results, also suggest the importance 
in future research, and in educational policy making as well, of understanding initial status 
differences and the cumulative effects of schooling among groups of students that differ by race and 
socioeconomic status. 
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APPENDIX A 



Table 2: Descriptive Statistics for African American (AA) Students 



AA 


Mean 


Variance 


Skewness 


Kurtosis 


Domain — Time 










Mathematics T1 


186.28 


256.24 


0.27 


-0.31 


Mathematics T2 


208.06 


358.00 


0.20 


-2.80 


Mathematics T3 


216.77 


553.50 


0.34 


-0.38 


Language T1 


189.72 


404.99 


0.25 


-0.22 


Language T2 


213.53 


726.40 


0.38 


-0.36 


Language T3 


226.84 


811.31 


0.28 


-0.42 



N=1 0,724 

Table 3: Descriptive Statistics for White Students 



WHITE 


Mean 


Variance 


Skewness 


Kurtosis 


Domain — Time 










Mathematics T1 


204.50 


411.12 


0.19 


-0.41 


Mathematics T2 


231.01 


541.72 


0.12 


-0.47 


Mathematics T3 


246.01 


687.09 


-0.17 


-0.69 


Language T1 


206.66 


574.12 


0.25 


-0.38 


Language T2 


237.53 


901.85 


0.02 


-0.59 


Language T3 


251.73 


959.18 


-0.02 


-0.63 



N=1 3,578 
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APPENDIX B 



Table 3: Longitudinal Data on Stratified Random Subsample of 27 Students with: (a) 3 Waves of 
Language Scores at Grades 4, 6 and 7 (b) 3 Waves of Math Scores at Grades 4, 6 and 7(c) 
Values of the Indicator (AA= African American; W=White). 



Subject ID 


Language 




Mathematics 


Race/Ethnicity 




Lang_4 


Lang_6 


Lang_7 


Math_4 


Math_6 


Math_7 




6983 


186.25 


220 


216 


202.0 


220 


196 


AA 


5979 


182.00 


200 


254 


155.0 


180 


227 


AA 


6241 


185.00 


194 


235 


182.0 


200 


212 


AA 


1579 


218.25 


257 


298 


224.5 


244 


260 


AA 


1033 


181.75 


196 


223 


181.5 


202 


198 


AA 


7061 


187.00 


194 


207 


190.5 


220 


238 


AA 


1995 


191.50 


211 


207 


178.5 


202 


196 


AA 


7848 


201.75 


258 


243 


186.0 


224 


215 


AA 


3199 


146.50 


178 


194 


165.5 


186 


204 


AA 


4770 


189.50 


226 


223 


195.0 


212 


236 


AA 


6537 


175.75 


210 


213 


214.5 


218 


226 


AA 


7820 


188.75 


202 


223 


221.0 


235 


249 


AA 


9612 


164.50 


193 


210 


182.0 


218 


224 


AA 


2597 


240.25 


292 


288 


233.5 


274 


294 


W 


4186 


237.75 


229 


210 


199.5 


254 


268 


W 


4696 


209.00 


248 


260 


204.0 


223 


242 


W 


1535 


222.25 


235 


229 


202.0 


206 


213 


W 


4431 


177.00 


202 


232 


218.0 


215 


264 


w 


7540 


208.50 


198 


232 


179.5 


188 


221 


w 


2179 


245.00 


274 


265 


215.5 


220 


221 


w 


9674 


201.00 


246 


218 


187.0 


196 


212 


w 


8021 


256.00 


266 


278 


214.5 


251 


262 


w 


1351 


194.00 


224 


254 


185.0 


217 


240 


w 


9364 


212.50 


228 


283 


217.5 


237 


280 


w 


1809 


166.25 


188 


190 


167.5 


168 


196 


w 


4158 


201.75 


199 


248 


200.0 


216 


239 


w 


7038 


219.50 


273 


232 


213.5 


228 


250 


w 
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APPENDIX C 
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Figure 1: OLS Fitted Trajectories Summarizing Linear Growth in Language between Grades 4 
and 7 for a Subsample of 1 3 Randomly selected African American Students whose associated 
Empirical Growth Records are provided in Table 4.5. 
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Figure 2: OLS Fitted Trajectories Summarizing Linear Growth in Language between Grades 4 
and 7 for a Subsample of 14 Randomly selected White Students whose associated Empirical 
Growth Records are provided in Table 4.5. 
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Figure 3: OLS Fitted Trajectories Summarizing Linear Growth in Mathematics between Grades 4 
and 7 for a Subsample of 1 3 Randomly selected African American Students whose associated 
Empirical Growth Records are provided in Table 4.5. 
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Figure 4: OLS Fitted Trajectories Summarizing Linear Growth in Mathematics between Grades 4 
and 7 for a Subsample of 14 Randomly selected White Students whose associated Empirical 
Growth Records are provided in Table 4.5. 
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